Russ Thurow USDA Forest Service Rocky Mountain Research Station
Claire McGrath Natural Resources Specialist, Columbia Hydropower Branch at NOAA Fisheries, West Coast Region, riverbio@yahoo.com
Kevin See Biometrician, Biomark Inc, Boise, ID, Kevin.See@merck.com
Salmon redd counts are widespread method to estimate the number of returning adult spawners. However, despite its prevalence in the Northwest, the reliability of redd counts is unknown. This work is focused on developing a statistical model to estimate the observer error in redd surveys, using a variety of covariates related to the habitat and the observer. We described three types of observer error:
Possible covariates in each error model are shown in Table 1. To make comparisons with AICc, the random effects must be identical across all models. Therefore, we ensured that the random effect of year was added to any model that didn’t have it.
Table 1: Possible covariates included in each observer error model.
| Type | Covariate | Air | Ground |
|---|---|---|---|
| Random | Reach | X | X |
| Random | Surveyor | X | |
| Random | Year | X | X |
| Fixed | ANNDist_log | X | X |
| Fixed | AveAge | X | X |
| Fixed | AveBadCond | X | |
| Fixed | AveCanopy | X | X |
| Fixed | AveContrast | X | X |
| Fixed | AveDepth | X | X |
| Fixed | AveOverlap | X | X |
| Fixed | AveSunny | X | |
| Fixed | AveWidth | X | X |
| Fixed | ExperienceCat | X | |
| Fixed | I(AveDepth^2) | X | X |
| Fixed | LYabund | X | X |
| Fixed | LYabund:PeakQ | X | X |
| Fixed | OthrDens | X | X |
| Fixed | PeakQ | X | X |
| Fixed | redd_dens | X | X |
| Fixed | Slope | X | X |
All covariates were z-scored, and all models were fit using the glmer or lmer functions from the lme4 package (Bates et al. 2015) in R software (R Core Team 2019). The amount of variation explained by fixed and random effects was calculated using the methods of Nakagawa and Schielzeth (2013). Using estimated predictions of the rates for omission (\(\hat{\omega}\)), commission (\(\hat{\eta}\)) and net error (\(\hat{\gamma}\)), we predicted the number of actual redds by either dividing the observed counts, \(c\), by estimates of net error, or by multiplying the observed counts by 1 - estimated rate of commission, and then dividing by 1 - estimated rate of omission.
We performed a cross validation by dividing each survey type data into 5 training datasets where 20% of the data was withheld for testing, and then fitting the naive and best AICc model formulations to the remaining data, and then using those fits to predict the error rates and true number of redds for each survey in the year that had been withheld.
\[ \begin{aligned} redds_{ne} &= \frac{c}{\hat{\gamma}} \\ redds_{om} &= c * \frac{1 - \hat{\eta}}{1 - \hat{\omega}} \end{aligned} \]
The observed error rates are showin in Figure 1.
Figure 1: Observed error rates.
The model coefficients of the full, best (by AICc) and model averaged models are shown in Table 2.
Table 2: Estimated coefficients for various observer error models.
| Survey | Resp | Covariate | avg | best | full |
|---|---|---|---|---|---|
| Ground | Com | (Intercept) | -0.962 | -0.961 | -0.961 |
| Ground | Com | ANNDist_log | 0.360 | 0.359 | 0.359 |
| Ground | Com | AveAge | -0.042 | -0.042 | -0.042 |
| Ground | Com | AveCanopy | 0.022 | 0.022 | 0.022 |
| Ground | Com | AveContrast | 0.018 | 0.018 | 0.018 |
| Ground | Com | AveDepth | 0.152 | 0.152 | 0.152 |
| Ground | Com | AveOverlap | 0.027 | 0.027 | 0.027 |
| Ground | Com | AveWidth | -0.137 | -0.137 | -0.137 |
| Ground | Com | ExperienceCat.L | -0.162 | -0.162 | -0.162 |
| Ground | Com | ExperienceCat.Q | 0.305 | 0.305 | 0.305 |
| Ground | Com | I(AveDepth^2) | 0.064 | 0.064 | 0.064 |
| Ground | Com | LYabund | 0.270 | 0.270 | 0.270 |
| Ground | Com | LYabund:PeakQ | 0.107 | 0.107 | 0.107 |
| Ground | Com | OthrDens | 0.223 | 0.223 | 0.223 |
| Ground | Com | PeakQ | -0.122 | -0.122 | -0.122 |
| Ground | Com | redd_dens | 0.102 | 0.102 | 0.102 |
| Ground | Com | Slope | 0.034 | 0.034 | 0.034 |
| Ground | Net | (Intercept) | -0.390 | -0.390 | -0.236 |
| Ground | Net | ANNDist_log | 0.260 | 0.258 | 0.269 |
| Ground | Net | AveAge | -0.059 | - | -0.059 |
| Ground | Net | AveCanopy | -0.017 | - | -0.017 |
| Ground | Net | AveContrast | 0.020 | - | 0.020 |
| Ground | Net | AveDepth | 0.035 | - | 0.035 |
| Ground | Net | AveOverlap | -0.004 | - | -0.008 |
| Ground | Net | AveWidth | -0.096 | - | -0.096 |
| Ground | Net | ExperienceCat.L | 0.322 | - | 0.322 |
| Ground | Net | ExperienceCat.Q | -0.170 | - | -0.170 |
| Ground | Net | I(AveDepth^2) | -0.002 | - | -0.002 |
| Ground | Net | LYabund | 0.263 | - | 0.263 |
| Ground | Net | LYabund:PeakQ | 0.177 | - | 0.177 |
| Ground | Net | OthrDens | 0.003 | - | 0.003 |
| Ground | Net | PeakQ | -0.003 | - | -0.003 |
| Ground | Net | redd_dens | 0.049 | - | 0.031 |
| Ground | Net | Slope | 0.024 | - | 0.024 |
| Ground | Omi | (Intercept) | -0.387 | -0.387 | -0.387 |
| Ground | Omi | ANNDist_log | -0.069 | -0.069 | -0.069 |
| Ground | Omi | AveAge | 0.174 | 0.174 | 0.174 |
| Ground | Omi | AveCanopy | 0.002 | 0.002 | 0.002 |
| Ground | Omi | AveContrast | -0.051 | -0.051 | -0.051 |
| Ground | Omi | AveDepth | 0.110 | 0.110 | 0.110 |
| Ground | Omi | AveOverlap | 0.098 | 0.098 | 0.098 |
| Ground | Omi | AveWidth | 0.025 | 0.025 | 0.025 |
| Ground | Omi | ExperienceCat.L | -0.746 | -0.746 | -0.746 |
| Ground | Omi | ExperienceCat.Q | 0.669 | 0.669 | 0.669 |
| Ground | Omi | I(AveDepth^2) | -0.049 | -0.049 | -0.049 |
| Ground | Omi | LYabund | -0.565 | -0.565 | -0.565 |
| Ground | Omi | LYabund:PeakQ | -0.736 | -0.736 | -0.736 |
| Ground | Omi | OthrDens | -0.031 | -0.031 | -0.031 |
| Ground | Omi | PeakQ | -0.233 | -0.233 | -0.233 |
| Ground | Omi | redd_dens | 0.245 | 0.244 | 0.244 |
| Ground | Omi | Slope | 0.380 | 0.380 | 0.380 |
| Survey | Resp | Covariate | avg | best | full |
|---|---|---|---|---|---|
| Air | Com | (Intercept) | -1.339 | -1.342 | -1.285 |
| Air | Com | ANNDist_log | 0.382 | 0.290 | 0.296 |
| Air | Com | AveAge | -0.206 | - | -0.052 |
| Air | Com | AveBadCond | -0.003 | - | -0.003 |
| Air | Com | AveCanopy | -0.041 | - | -0.037 |
| Air | Com | AveContrast | -0.060 | - | -0.007 |
| Air | Com | AveDepth | -0.191 | - | -0.191 |
| Air | Com | AveOverlap | -0.136 | -0.136 | -0.143 |
| Air | Com | AveSunny | 0.332 | - | 0.291 |
| Air | Com | AveWidth | 0.164 | - | 0.164 |
| Air | Com | I(AveDepth^2) | 0.050 | - | 0.050 |
| Air | Com | LYabund | 0.730 | - | 0.747 |
| Air | Com | LYabund:PeakQ | 0.679 | - | 0.701 |
| Air | Com | OthrDens | 0.069 | - | 0.072 |
| Air | Com | PeakQ | -0.167 | - | -0.158 |
| Air | Com | redd_dens | -0.204 | -0.198 | -0.138 |
| Air | Com | Slope | 0.022 | - | 0.022 |
| Air | Net | (Intercept) | -0.402 | -0.401 | -0.215 |
| Air | Net | ANNDist_log | 0.283 | 0.303 | 0.238 |
| Air | Net | AveAge | -0.179 | - | -0.179 |
| Air | Net | AveBadCond | 0.003 | - | 0.003 |
| Air | Net | AveCanopy | -0.072 | - | -0.070 |
| Air | Net | AveContrast | 0.077 | - | 0.038 |
| Air | Net | AveDepth | -0.061 | - | -0.062 |
| Air | Net | AveOverlap | -0.102 | - | -0.105 |
| Air | Net | AveSunny | 0.137 | - | 0.060 |
| Air | Net | AveWidth | -0.040 | - | -0.041 |
| Air | Net | I(AveDepth^2) | 0.026 | - | 0.026 |
| Air | Net | LYabund | 0.463 | - | 0.465 |
| Air | Net | LYabund:PeakQ | 0.445 | - | 0.447 |
| Air | Net | OthrDens | -0.080 | - | -0.060 |
| Air | Net | PeakQ | -0.084 | - | -0.083 |
| Air | Net | redd_dens | 0.010 | - | -0.051 |
| Air | Net | Slope | -0.052 | - | -0.052 |
| Air | Omi | (Intercept) | -0.565 | -0.565 | -0.522 |
| Air | Omi | ANNDist_log | -0.202 | - | -0.206 |
| Air | Omi | AveAge | 0.507 | 0.507 | 0.591 |
| Air | Omi | AveBadCond | -0.012 | - | -0.012 |
| Air | Omi | AveCanopy | 0.046 | - | 0.046 |
| Air | Omi | AveContrast | 0.014 | 0.014 | 0.010 |
| Air | Omi | AveDepth | 0.374 | - | 0.374 |
| Air | Omi | AveOverlap | 0.287 | 0.287 | 0.180 |
| Air | Omi | AveSunny | -0.050 | - | -0.050 |
| Air | Omi | AveWidth | -0.096 | - | -0.096 |
| Air | Omi | I(AveDepth^2) | -0.097 | - | -0.097 |
| Air | Omi | LYabund | -0.457 | - | -0.457 |
| Air | Omi | LYabund:PeakQ | -0.732 | - | -0.732 |
| Air | Omi | OthrDens | 0.108 | - | 0.108 |
| Air | Omi | PeakQ | 0.157 | - | 0.157 |
| Air | Omi | redd_dens | -0.085 | - | -0.091 |
| Air | Omi | Slope | 0.308 | - | 0.308 |
The relative importance of each covariate in each model is shown in Figure 2, while the amount of the variance explained by fixed and random effects in the best AICc model is shown in Figure 3. Observed versus predicted rate plots are shown in Figures 4, 6 and 8.
Figure 2: Relative importance of each covariate in ground-based observer error models
Figure 3: How much variance in the model response is explained by the fixed and random effects in the best AICc model.
Figure 4: Observed versus predicted rates of omission using model averaged predictions, the single best model, and the naive model (only random effects).
Figure 5: Correlations between observed omission rates and three model predictions (model averaged, single best and naive).
Figure 6: Observed versus predicted rates of commission using model averaged predictions, the single best model, and the naive model (only random effects).
Figure 7: Correlations between observed commission rates and three model predictions (model averaged, single best and naive).
Figure 8: Observed versus predicted rates of net error using model averaged predictions, the single best model, and the naive model (only random effects).
Figure 9: Correlations between observed net error rates and three model predictions (model averaged, single best and naive).
We examined the bias in estimates rates, using both the best (by AICc) model and the naive model (only random effects).
For ground-based surveys, both methods provided fairly unbiased estimates of the true number of redds (Figure 10), although the omission/commision models had slightly higher absolute and relative bias (Table 3).
Figure 10: Boxplots of absolute and relative bias for each type of predictive model.
Table 3: Summary statistics of predictions of total redds from leave-one-out cross validation using the net error and the omission/commission models.
| Model | Median # Obs. Redds | Median # True Redds | Median Adjustment | Median Abs. Bias | Median Rel. Bias (%) | RMSE |
|---|---|---|---|---|---|---|
| Best Net Error | 36 | 38 | 5.3 | -0.1 | -0.6 | 24.1 |
| Best Omis / Comm Error | 36 | 38 | 4.3 | -0.6 | -2.1 | 17.7 |
| Naive Net Error | 36 | 38 | 5.7 | 0.0 | 0.1 | 24.0 |
| Naive Omis / Comm Error | 36 | 38 | 4.0 | 0.3 | 1.0 | 17.5 |
| Observed | 36 | 38 | - | -2.0 | -8.0 | 18.9 |
Figure 11: Observed number of true redds vs. leave-one-out cross validated predicted redds based on either the best AICc or naive versions of the net error or omission/commission models. Dashed line is the 1-1 line, while solid line with gray error ribbon is the best fit linear model to these data.
The relative importance of each covariate in each model is shown in Figure 12, while the amount of the variance explained by fixed and random effects in the best AICc model is shown in Figure 13. Observed versus predicted rate plots are shown in Figures 14, 16 and 18.
Figure 12: Relative importance of each covariate in ground-based observer error models
Figure 13: How much variance in the model response is explained by the fixed and random effects in the best AICc model.
Figure 14: Observed versus predicted rates of omission using model averaged predictions, the single best model, and the naive model (only random effects).
Figure 15: Correlations between observed omission rates and three model predictions (model averaged, single best and naive).
Figure 16: Observed versus predicted rates of commission using model averaged predictions, the single best model, and the naive model (only random effects).
Figure 17: Correlations between observed commission rates and three model predictions (model averaged, single best and naive).
Figure 18: Observed versus predicted rates of net error using model averaged predictions, the single best model, and the naive model (only random effects).
Figure 19: Correlations between observed net error rates and three model predictions (model averaged, single best and naive).
We examined the bias in estimates rates, using both the best (by AICc) model and the naive model (only random effects).
For air-based surveys, both methods provided estimates of the true number of redds that were biased high (Figure 20). However, the net error models had lower absolute and relative bias, as well as a smaller root squared mean error (RMSE) (Table 4).
Figure 20: Boxplots of absolute and relative bias for each type of predictive model.
Table 4: Summary statistics of predictions of total redds from leave-one-out cross validation using the net error and the omission/commission models.
| Model | Median # Obs. Redds | Median # True Redds | Median Adjustment | Median Abs. Bias | Median Rel. Bias (%) | RMSE |
|---|---|---|---|---|---|---|
| Best Net Error | 32 | 38 | 8.4 | 0.5 | 0.5 | 26.6 |
| Best Omis / Comm Error | 32 | 38 | 7.4 | 1.3 | 7.4 | 20.5 |
| Naive Net Error | 32 | 38 | 6.3 | -0.2 | -0.9 | 29.6 |
| Naive Omis / Comm Error | 32 | 38 | 6.8 | -0.9 | -3.0 | 25.5 |
| Observed | 32 | 38 | - | -6.0 | -18.2 | 28.7 |
Figure 21: Observed number of true redds vs. leave-one-out cross validated predicted redds based on either the best AICc or naive versions of the net error or omission/commission models. Dashed line is the 1-1 line, while solid line with gray error ribbon is the best fit linear model to these data.
Bates, D., Mächler, M., Bolker, B., and Walker, S. 2015. Fitting Linear Mixed-Effects Models Using lme4. Journal of Statistical Software, 67(1): 1–48.
Nakagawa, S., and Schielzeth, H. 2013. A general and simple method for obtaining r2 from generalized linear mixed-effects models. Methods in Ecology and Evolution 4(2): 133–142. Wiley Online Library.
R Core Team. 2019. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria.
Biometrician, Biomark, Inc., Kevin.See@merck.com↩︎
Natural Resources Specialist, Columbia Hydropower Branch at NOAA Fisheries, West Coast Region, riverbio@yahoo.com↩︎
USDA Forest Service Rocky Mountain Research Station↩︎